(1) Field of the Invention
The present invention relates to the field of computer systems. More specifically, the present invention relates to the field of memory addressing methods and systems within computer systems.
(2) Prior Art
Large computer systems today contain memory management hardware and software which provide processes with access to a defined range of physical memory (e.g., hardware volatile memory). Operating systems of the computer system utilize memory stored data structures which define and maintain virtual address memory spaces which can be made private to each process. The virtual memory space is then used transparently by the process to access data that is stored within the physical address memory space. The above data structures provide a mapping or "translation" between the addresses of the virtual memory space used by the process and the addresses of the physical memory space used by the computer system's hardware memory management systems which maintain the data in the physical memory. A typical computer system can provide respective memory mappings for as many as 500 or more processes simultaneously.
One prior art memory management system is shown in FIG. 1A. Two virtual address memory spaces are shown as 10a and 10b which correspond to a process 1 and a process 2, respectively. Also shown is a large physical memory address space 20. The physical memory address space 20 is typically realized in volatile memory (e.g., random access memory). According to the memory management system of FIG. 1A, within a 32-bit operating system, the allowed size of each virtual memory space 10a and 10b is approximately 2 gigabytes (GB), with another 2 GB of memory space being reserved for the operating system. The operating system allocates a working set of physical memory to each virtual address space 10a, 10b. Working set 22a is allocated for virtual memory space 10a and working set 22b is allocated for virtual memory space 10b. In a 32-bit operating system, the typical size allowed for a given working set is about 10-20 megabytes (MB), however, in some database applications a maximum of 200 MB can be reserved. In a prior art 32-bit operating system, a maximum of 4 GB of physical memory can be addressed. Therefore, the size of the working set is restricted by the operating system due to fairness concerns so that each process has sufficient access to the 4 GB of physical memory.
A working set is divided into pages of physical memory, e.g., one page 24a is shown for working set 22a and one page 24b is shown for working set 22b. The size of a page is programmable, but one typical size is 8 kilobytes (KB). When a process accesses data, it utilizes addresses within its virtual address space which are converted by a page table data structure into addresses within pages of the process's working set within the physical memory space 20. The operating system uses the page tables of each process to map pages of physical memory space 20 into the virtual address space (10a, 10b) that the process can access.
FIG. 1B illustrates three exemplary page table data structures 30a, 30b and 30n for process 1, process 2, and process n, respectively. The page table data structures 30a, 30b, and 30n are stored in the operating system's address space and maintain the mapping between a process's virtual addresses and the addresses of pages within the physical memory space 20 (FIG. 1A). For each process, virtual addresses are received (e.g., over lines 32a, 32b, and 32n) and the appropriate page address of the physical memory space are output by the tables (e.g., over buses 34a, 34b, and 34n). Since many processes can operate simultaneously (e.g., 500 or more), many page tables can be simultaneously accessed and need to be simultaneously maintained. As discussed further below, in order to share a block of physical memory across several processes, each process requires its own page table to provide the proper address mapping into the shared memory.
In the addressing mechanism of FIG. 1A and FIG. 1B, many more physical memory pages can be mapped by a process than fit within the process' working set. In other words, the virtual memory size in the prior art memory addressing system, for a single process, is very much larger than the allocated working set size. In this case, as additional physical memory pages are needed by the process, they are retrieved from a non-volatile memory storage (e.g., disk 104 of FIG. 1A) and copied into the working set 22a, 22b for that process, overwriting other pages that might currently exist within the working set 22a, 22b. Conversely, if a page of a working set 22a, 22b is not currently required, it is stored in the non-volatile storage 104 to free up space within the working set 22a, 22b for other information. The above "swapping" of pages to and from the non-volatile storage 104 and the volatile working sets 22a, 22b is called "paging." The smaller the working set size, the more disk input/output accesses are required to flow data into and out of the working sets 22a, 22b. Therefore, small working sets reduce the overall performance of the prior art memory addressing mechanism.
To maximize system performance, database processes within a database system require access to shared global memory sections (also called global memory caches) which are shared by all processes. One prior art mechanism utilized for this purpose is a process based global section. Process based global sections are mapped into the address space of each process that requests access to the global section, and thus are subject to the address space management policies and restrictions enforced by the operation system memory management subsystem. For instance, in the prior art, the number of physical pages owned by any one process is limited to the size of its working set or under another quota system. As discussed above, a working set is typically limited to 200 MB in size (400K pages of 2 KB) on 32-bit systems and 520 MB in size (65K pages of 8 KB) on 64-bit systems.
Small working sets increase paging. Paging requirements of the memory management subsystem reduce system performance when using the global sections of the prior art. For instance, accessing pages of the global section of the prior art that are not currently mapped in a process' working set requires mapping of the page into the process to make its individual PTE valid and further requires reading of the page from its backing storage device (e.g., from a hard disk). Each of these tasks reduces system performance. Further, global sections of the prior art require file backing storage, which can use a system page file or an image file. This means that dedicated disk space is needed for essentially all pages that are mapped that exceed the working set restrictions (e.g., to store what is not currently in the working set) thereby also reducing performance, especially for working sets that are very small. In addition, ejecting pages from the process' working set can result in writing the page back to its backing storage device, again reducing performance. What is needed is a system for providing shared memory that is not limited by the above small working set sizes and thereby is not subject to as much paging to and from disk storage.
Another disadvantageous policy of the operating system memory management subsystem is that process level page table entries (PTEs) of the global section are not shared between processes. In a prior art shared global section, a large amount of physical memory is required to maintain the mapping of the shared global section between the processes. Accordingly, each process requires its own PTE data structure to share the global section. Specifically, the physical memory required to store these replicated data structures is typically 8 bytes per PTE times the global section size in pages times the number of processes mapping to the global section. For example, consider 500 processes times (2 GB global section/8K page size) times 8 (bytes per PTE). This equals approximately 1 GB of operating system memory storage for the PTEs required for a 2 GB process based global section. Therefore, in a 32-bit operating system, the 1 GB memory used by the required PTE data structures consumes one quarter of the memory space available (4 GB) to the computer system's operating system. Such large memory usage for memory mapping overhead is not an effective use of memory resources. What is needed is a system for providing shared memory mapping that does not require such large amounts of memory storage to maintain PTE data structures.
The above performance and system memory consumption disadvantages are not alleviated or reduced within computer systems that contain 64-bit operating systems and/or 64-bit directly addressable physical memory spaces under the prior art addressing mechanism. Indeed, within these systems, the above performance and system memory consumption disadvantageous are exacerbated because the required sizes of the page table data structures per process are increased. Moreover, although 64-bit addressable physical memory provides a substantial amount of volatile memory storage, the prior art addressing mechanism still limits the size of the working set to 200 MB or 520 MB per process. This is especially true for 32-bit operating systems having 64-bit directly addressable physical memory. Therefore performance disadvantages result due to heavy disk I/O operations. For 64-bit operating systems having 64-bit directly addressable physical space, the larger limiting factor concerns the immense size of the page table data structures within the operating system's address space.
Accordingly, what is needed is an addressing system that provides a large amount of shared global memory space across many processes without the limitations described above. Specifically, what is needed is an addressing system that provides a large amount of shared global memory space to many processes and is not limited by the above working set sizes so it thereby offers efficient performance. What is further needed is an addressing system that provides a large amount of shared global memory space to many processes and does not consume large amounts of system virtual memory for storing large numbers of PTE data structures. The present invention memory addressing system provides the above features and others not necessarily recited above but which will become clear within discussions of the present invention herein.